The Advantages of a Biweight Metric in Clustering Microarray Data

Distance metrics are often the backbone of clustering algorithms. Yet certain distance metrics, such as one based on Pearson’s correlation, are sensitive to outliers. Microarray data tend to have outlying data points. Hence, we may intuitively believe metrics like one based on Pearson’s correlation may not be appropriate for clustering microarray data. Hardin, et al. (2007) show a metric based on Tukey’s biweight estimate of multivariate scale and location to be more robust than a metric based on Pearson’s correlation. The goal of this paper is to find a way to evaluate whether a metric based on Tukey’s biweight will create more valid clusters (based on known partitions in microarray data) than a metric based on Pearson’s correlation. We define an extreme outlier to be a data point that is sufficiently outlying with respect to the relationship between two genes. When an extreme outlier is removed, a metric based on Pearson’s correlation will often dramatically change its measurement of correlation, while a metric based on Tukey’s biweight will often display very little change in its estimate of correlation [11]. We consider a “robust” cluster to be a cluster that would be clustered the same with the removal of extreme outliers from gene pairings as without the removal. Our results suggest that a metric based on Tukey’s biweight will create more “robust” clusters than a metric based on Pearson’s correlation. As our clustering algorithm, we use Partitioning Around Medoids (PAM) [14].